Lessons from Giant-Scale Services

W eb portals and ISPs such as AOL, Microsoft Network, and Yahoo have grown more than tenfold in the past five years. Despite their scale, growth rates, and rapid evolution of content and features, these sites and other “giant-scale” services like instant messaging and Napster must be always available. Many other major Web sites such as eBay, CNN, and Wal-Mart, have similar availability requirements. In this article, I look at the basic model for such services, focusing on the key realworld challenges they face — high availability, evolution, and growth — and developing some principles for attacking these problems. This is very much an “experience” article. Few of the points I make here are addressed in the literature, and most of my conclusions take the form of principles and approaches rather than absolute quantitative evaluations. This is due partly to my focus on high-level design, partly to the newness of the area, and partly to the proprietary nature of some of the information (which represents 15-20 very large sites). Nonetheless, the lessons are easy to understand and apply, and they simplify the design of large systems. The Basic Model I focus on “infrastructure services” — Internet-based systems that provide instant messaging, wireless services such as iMode, and so on. These services primarily reside remotely from the user, although they might include local access software, such as a browser. My discussion is limited primarily to single-site, single-owner, well-connected clusters, which may be part of a larger service, as in the case of e-mail servers. I do not cover wide-area issues such as network partitioning, low or intermittent bandwidth, or multiple administrative domains. There are many other important challenges that I do not address here, including service monitoring and configuration, network quality of service (QoS), security, and logging and log analysis. This article is essentially about bridging the gap between the basic building blocks of giant-scale services and the realworld scalability and availability they require. It focuses on high availability and the related issues of replication, graceful degradation, disaster tolerance, and online evolution. Database management systems (DBMs) are an important part of many large sites,